This invention relates to an object recognition and tracking system.
Systems have been proposed for counting objects such as people passing along a corridor in for instance a shopping centre but such systems are not adapted for recognition or discrimination of certain or different types of objects and their position and to track such objects over time.
It is the object of this invention to provide an object recognition and tracking system.
In one form therefore, although this may not necessarily be the only or broadest form invention is said to reside as an object recognition and tracking system including at least one detector array, the or each detector array including a first type of detector and a second type of detector, the first type of detector being adapted to detect the presence of an object in a detector space and to define the object""s position, the second type of detector being adapted to use the defined object position to identify or recognise the object or to count or to record the presence of the object if it is of a selected type.
The presence of an object as determined by the first detector may be detected by a change in contrast boundaries or by a change in pattern or by movement and hence to detect objects that move within the detector space.
The detectors may be adapted to use a portion of the electromagnetic spectrum selected from radar, microwave, radio frequency, infrared, microwave, millimetre wave, light or other portion of the electromagnetic spectrum or to use sonar or some other detection system.
The first type of detector and the second type of detector may be of the same type or they may be of different types. Hence the first type of detector may be a low resolution video system and the second type of detector may be a radio frequency detection system.
Where the first detector and the second detector are in effect the same detector the second detector may operate at a higher resolution than the first detector or be a specific identifying type.
In a preferred form of the invention the first type of detector maybe of an insect eye type. That is, a detector which is adapted to detect change in contrast between adjacent pixels in a display space rather than to identify objects absolutely. In such a form of the invention the second type of detector may be a digital signal processor detector such as a digital video detector. Hence the first type of detector acts to identify a specific region of interest within a detector space and then the digital signal processor detector need only identify the object by looking in that particular region of the space rather than by processing a whole video image of the detector space. By this means considerably less processing power is required and greater efficiencies can be obtained.
The first detector and the second detector could be the one device with the second detector acting as a second stage of the process.
Hence, in an alternate form of the invention the first and second types of detectors may be provided by a single video detector the observed image of which is handled in two different ways. In a first way the observed image may be digitised and stored in successive memory locations and then sequential images may be scanned to detect changes in the digitised image over successive frames which would indicate a moving contrast boundary. This thereby defines a moving object in the detector space and the same digital image can then be analysed in that particular region of the detector space by the second type of detector. The second type of detector could use the subsequent or previous frames to improve the recognition or result with respect to the object of interest.
If the recognition process does not reach predetermined or artificially learned threshold levels of reliability then further stored or downloaded algorithms could be used in an iterative process until the results are improved.
Whilst the first detector type is used to identify an object of interest, the second type of detector is used for recognition purposes. Each object possesses a number of characteristics such as mass, height, shape, contrast, brightness, colour, patterns, velocity, heat, reflectivity, signature and many others. Each characteristic on its own does not necessary lend itself as being a characteristic that can be used to recognise an object by itself, and some characteristics lend themselves to be more easily identified than other characteristics. A preferred part of this invention is the analysis of the most recognisable of characteristics first, then by using other characteristics and successive frames in time, at each step in the iterative analysis process, the system improves its likelihood of recognising an object with greater accuracy. The determination of which characteristics are analysed first and the sequence of which they are analysed is a most preferred part of this invention.
In an alternate form of the invention, the system may be built with artificial intelligence to make these decisions without external intervention. In this case, the system determines which characteristic would be analysed first. As time progresses, the system may identify new characteristics or a new sequence of characteristics that may better identify an object of interest and therefore become self-improving. In this case, self improved procedures, or algorithms are stored with an existing memory resident library for future reference. The system would use artificial intelligence analysis to decide which algorithm is more suited by its success and speed of recognition using probability analysis.
Generally, therefore, the invention discloses a system which may be broken down into a number of stages. The first stage of the process would include an object detection engine to capture and load the digitised image sequences from the hardware into an appropriate digital format for the object recognition engine. The first type of detector may identify the boundary of an object as determined over successive frames as the object moves across the detector space. The second type of detector, which could form a second stage of the system would be an object recognition engine used to, firstly, convert certain characteristics into useful mathematical images or algorithms, and secondly, may be used to perform and record results over time necessary to analyse specific characteristics of the objects in question. The characteristics under analysis may include but would not be limited to one or more of items such as the mass, height, shape, colour, pattern, contrast, texture, heat, reflectivity, frequency, velocity, signal radiation, signature etc. Each characteristic of the detected object is given a chance of probability of being an object of interest. To obtain higher levels of accuracy, successive digitised images may be analysed over time using stored procedures, stored libraries, analysis algorithms and an iterative analysis process which can manage the observation of multiple characteristics from multiple objects in real-time.
Each analysis iteration increases (or decreases as the case may be) the chance of a detected object being an object of interest. If the level of probability reaches a predetermined level of acceptance or threshold level, the object is recognised as an object of interest. To improve the probability analysis, additional kinds of the second type of detector could be included within a single detector space used to view different characteristics of an object of interest to improve the probability of recognising an object.
Additional detectors types may be positioned in different locations and facing a different angles but mounted to view the same detector space to improve the chance of differentiating objects, placing or positioning objects in a three dimensional space or locating an object which would otherwise be obscured by the use of only one detector type.
The second type of detector may be adapted to interrogate an interrogatable device associated with the object. In one such embodiment the first detector may be adapted to observe that a an object is moving within a detection space and then the interrogation device of the second detector is adapted to interrogate an rf signature chip such as a smart card chip or transponder to provide a positive identification of the object and its passage through the detection space.
The object recognition and tracking system may further, include multiple detector arrays, multiple detector types and multiple detector spaces.
The detector array may be arranged to communicate with adjoining detector arrays in a multiple detector space to pass on or receive information it has received or learnt. This transmission of information could be conducted over a closed network such as an intranet or over an open network such as the internet. Decisions or information would be passed onto successive detectors or other third party devices requiring such information to activate as necessary programmed actions. The detectors may make decisions, give instructions or initiate some other form of activity without human intervention.
A third stage of the system may be performed by the tracking engine. Having recognised an object of interest, the tracking engine may perform such tasks as grid initialisation, 2D or 3D mapping, allocation of a tag, tag tracking and tag handover in successive detection spaces. This requires substantially less processing power than the object recognition engine as the identified and tagged object moves across one detector space into another. If an object appears in a successive detector space which was not previously detected because of an obscurity from another object, the process may be back tracked through all possible recorded image frames and the records may be adjusted accordingly. The entire system is updated with this new information. This substantially increases the accuracy of the system. From this point in time onwards, the object may be tracked until it moves out of all detection spaces. Having recognised the type of the object required to be tracked, the position of each object of interest is subsequently mapped onto a universal grid over time. As the processing power required for tracking is substantially reduced in successive detector spaces, multiple objects may be tracked with a high level of accuracy and reliability.
In a preferred form of the invention there may be multiple detector arrays, multiple detector types and multiple detector spaces in two dimensional or three dimensional space and the recognition and tracking system may include means to communicate with adjacent detector arrays to hand over tracking of a selected object from one detector space to an adjacent detector space and to record the movement of the object over time.
By this means the system may be able to track moving objects of a selected type within a much larger area than a single, detector space. This could be of use for instance if the requirement is to tracks objects throughout a shopping centre as well as individual detector spaces such as a display area.
It will be noted that as objects which are not moving in the detector space such as items of furniture or other fixed objects are not observed by the first type of detector which only observes moving contrast boundaries or changes for instance between adjacent frames of a video image and therefore are not tracked or observed by the second detector. This would save considerable processing power.
It will be realised that if the detector array is provided in the ceiling of a building looking substantially downwards in a corridor for instance an object when directly below the detector will have a certain profile but when the object is moving through the detector space the observed profile from a fixed detector array will vary with the angle observance and hence in one preferred form of the invention the processing arrangement to identify a moving object in the detector space may include means to take into account the angle of the object from the detector array or the movement of objects off centre within the space.
The object recognition and tracking system may have its or some of the detector arrays mounted to a side of its respective detection space such as on a wall defining a boundary of a detection space or in a corner of a detection space.
The object recognition and tracking system of the present invention may be used for counting people in supermarkets or the like or for traffic travelling along a road, for security monitoring and surveillance applications and to determine human preferences when walking through a shop or display area or to determine how many people are located within a three dimensional space such as a room or region.
The object recognition and tracking system of the present invention may be used to track seemingly random movements to determine patterns or trends over time.
The detector system of the present invention may be used as a surveillance system with a human interface to create a readable image of what the detector is viewing such as a video camera would.
It should be noted that the detector array could be mounted on a moving device such as a motor vehicle and its observations being related to stationary objects such as street signs or stop lights. It may be used to detect the presence of pedestrians.
This then generally describes the invention but to assist with understanding reference will now be made to the accompanying drawings which show a preferred embodiment of the invention.
In the drawings:
FIG. 1 shows a detector space with some objects within the detector space;
FIG. 2 shows the objects and the regions which are observed by the first detector type;
FIG. 3 shows the detector region and the detection done by the second type of detector;
FIG. 4 shows an enlarged detector space with multiple detector arrays and objects moving through the detector space;
FIG. 5 shows a more realistic position of objects observed by the first type of detector in a detector space;
FIG. 6 shows the objects of FIG. 5 observed by the second type of detector;
FIG. 7 shows a schematic diagram of a recognition and tracking system according to the present invention;
FIG. 8 shows a schematic diagram of a detection system according to an alternative embodiment of the present invention;
FIG. 9 shows a schematic diagram of a detection system according to an alternative embodiment of the present invention;
FIG. 10 shows a schematic diagram of a detection system according to an alternative embodiment of the present invention; and
FIG. 11 shows a schematic diagram of a detection system according to an alternative embodiment of the present invention.
FIG. 12 shows a schematic diagram of a detection system according to an alternative embodiment of the present invention using a transponder signature identification system.